Sparsity-Based Criteria for Entropy Measures

The complexity of a signal defines the compressibility of its coefficients under appropriate basis. This relation complexity-compressibility suggests a connection between the functions used to measure both of the signal’s properties. A measure of entropy quantifies the degree of complexity of a signal. In a analogous way, a measure of (non-strict) sparsity quantifies the degree of compressibility of a signal projected in a different space. Hence both families of measures, sparsity and entropy, should follow similar criteria. In this paper, a set of intuitive criteria for sparsity measures is collected. Using this list model, a set of criteria for entropy measures is proposed. Then, two simple sparsity and entropy measures satisfying the criteria are constructed. We are looking for simplicity in the sense that these measures allow an online implementation. The definition of both measures is based in a simple comparison with reference signals. Further, this leads to surprising simplifications. Among the implications of this work, we present an novel understanding of complexity which leads to a re-interpretation of entropy measures.